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.ABSTEACT . . • 

Performance of 237 students in ten eighth^gra^e 
mathematics classes was assessed to measure their ah§B.lity to soly^ 
(1) problems which involve only computation, and (2) problems which 
inv.olve computation and recognition of the, relevance ' of / a particular 
loathematical idea indispensable to dthe solution of the /problem. 
Pretest and/or posttest data were coljlected for five di|f£ter6nt -sets 
of experimental conditions represented in the ten clarases. The 

'effects of the. various sets of experimental conditions jwfere measured 
by two different forms of two specially constructed te3ts. The study 
provides strong support for the propositioti that skills in applying 

' • mathem,atical ideas can be improved by learning procedure^ that are 
rich in o.pportunities for application at appropriate levels of 
complexity for each student. (Author/JBW) 
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IMPROVING SKILL Ifl APPLYING MATHEMATICAL IDEAS: ^ 

* A Preliminary Report on the 

Instructional, Gaining Program aC Pelham Middle School in Detroit 

«. 

Layman E. Allen and Joan Rosd • . 

UnlversityVof MlchigatfT 

* t 

Performance of 237 students in ten 8th-grdde mathematics classes ( 
was assessed to measure their ability to solve problems which involve 
only computation and problems which involve both computation and 
recognition of the relevance of ^a particular mathematical idea which 
i8 indispeirsable to -the solution of thja problem. Pretests and post- 
tests focused upon 21 ymathematical ideas which are presented in the 
. present series of IMP (Instructional ^ath Play) Kits, 16-page 

pamphlet simulations of a computer programmed to play EQUATIONS like 
a good teacher^ rather than like a good player. 

Results show that tHi combination of playing EQUATIONS over a* 
two-year period and. then working intensively with the IMP Kits for^ 
two weeks enables students to apply mathematical ideas (in the ^senae 
studied in this experiment) better than any df ,the other four sets 
of conditions do: better than just playing EQUATIONS alone, better * 
than playing .EQUATIONS and being tkugbt explicitly by the teacher 
7 the 21 ideas presented in the IMP I^its, better tUan being taught^ 

th^2l ideas in an ordinary traditional mathematics class, and better 
than being in an ordinary traditional class without any special 
teaching of the ideas — and, furthermore, better in each case by a 
highly conservative test at an extreme level of significance (.0001) • 
• 'At the beginning of ^ the experiment the IMF Kit group was clearly 

performing at a higher level of achievement than were the other 
groupSn. This superior performance, seems linked to their two-year 
experience in playing EQUATIONS. Available evidence indicates that 
the EQUATIONS sitid nonEQUATIONS groups were not different upon their 
entry to 7t4i^grade, but that after two years of different experience 
with respect to whe.ther or not they played EQUATIONS, the EQUATIONS 
group was significantly better in both computing with and applying* 
mathematical, ideas. The emphasis'^ of the IMP Kit experience is 
cleatly in the direction of improving skills in applying mathematical 
ideas, although it does improve both computation and application* 
III terms of reducing the difference between understanding a 
mathematical idea in the ^sense of computing correctly wiljh it and 
understanding it ^ti the sense of being able to apply it in a context 
• where it must be recognised td' be relevant, playing through the IMP 
Kits clearly Is more effective than any ^ of the other^ethods trie/J 
in this experitnent (except possibly playing EQUATIONS alone, whe#e' 
the effect is in tffe right direction but not significantly so).- 
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IMPROVING SKIlL IN APPLYING MATHEMATICAL IDEAS: 
* A Preliminary Report; on the 

Instructional Gaming Program at Pelham Middle School in Detroit* 

Laysan E. Allen and Joan Ross 
University of Michigan 



INTRODUCTION 

Educators working with the EQUATIONS game, and associated 
materials quickly become aware that applying mahhematical ideas 
is much 'more dif fic]ult - than merely computing with those, ideas. 
Of cpurse, "applying" an idea is a somewhat vagi^ notion; this 
preliminary report will addressed to ^e aspect of application 
— namerly, recognition that, an idea is indispensably relevant to 
the solution of a probrem.. Consider the following pair of problem^: 
the first is fche C-type, (computation) , and the second is the R-type 
(relevance). ' ' 



1. 6- (1-3) 



B C \ Y«s No 



2. - - 1 3 6 8 



\ By writing an X in the Yes or No cojiimri, indicate whether 

or not all of the numbers and operations in Column A can 
be appropriately ordered and^^ouped (inserting parentheses 
' wherever necfessary) to form an expresBlbti equal to the 
number in Column B. If your answer is^Yes, write that 
expression in Column C. / 

The indispensably relevant idea for solving each of the problems 
is the subtraction of negative numbers. IfKthe C-rtype problem, 
the very statement of the problem clearly ana eiq^licitly indicates 
that subtracting a negative ^number is ii^olved. That is neither so 
clearly nor so explicitly cued in the statement of the R--type 
problem. Those who understand how to subtract negatljr^ numbers 
can easily do the first problem correctly. But many of those who 
can solve a C-type problem involving subtraction of jcieigafclve ^ 
numbers fail to solve q corresponding R-€y|e j)roblein Involj^tftg^ 
negative numbers. In general (in the groups ye have: studied) / 
about two-thirds of ^fhose who- solve C-type problems fall to solve 
a corresponding R-type problem that involves the sainjK idea. The 
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R-type problem Ilb considerably harder^ than the C-typ6 In this' 
example because the student must recognize from less clear and less 
explicit cues thMt subtracting negative numbers Is an Indispensably 
Relevant Idea for solving the problem. "Understanding" an Idea v 
In the -R-sense (being able* to solve R-type probleins) Includes 
understanding It In the C-sense, but It also Involves something 
more. R-sense understanding includes'the capability of selecting" 
from among a storehouse of ideas understood in the C-sense, those 
thato are indispensably relevant f ot solving a particular problem. 
The question to ^hlch this study is addressed is whether skills in 
applying mathematical ideas can be Improved by learning procedures 
that emphasize exposure to situations that are rich in opportunities 
f 013^ such application at levels of complexity that are appropriate 
for each learner. 

The EQUATIONS Game and the IMP (Instructional Math Play) Kits 

Ihe rules that define the EQUATIONS game establish a problem- 
generating and pisoblem-solving interaction between small groups of 

. students, an interaction that can easily be controlled to provide 
a highly individualized learning experience for each of the 
participants. It Is a RAG (Resource Allocat"ion .Game) where the 
resources involved a^e n^athematlcal ideas . (Fbr details, see 
Allen, 1972.) The IMP Kits are 16-page pamphl^t-sltaulatlons of 
a compjiter playing EQUATIONS wher6 the computer is programmed to 

.play like a good teacher, rather than like, a goocji player. Each kit 
presents a lesson on one irtathematlcal idea. There •are at present^ 
five versions of each of 21 ideas, or a total of 105 kits. (For 
details, see Allen & Ross, 197A.) - 

\ \ ^ \ " ] \ METHODS — ^ 

Sutijectg ^ 

All of the students in each of ten of/ the fourteen eighth- ^ 
grade clas^6 in mathematics at Pelham Middle School^ participated 
. in the stud^. The ten classes w^re chosen to Include all four of 
the classes in which the EQUATIONS game had been used during the 
prior two ye^Trs as p&rt of the regular Instructional program In 
mathematics and two other classes of each of the three participating 
. teachers. Pretest and/or post test data were collected on 237 of 
the students enrolled in these ten classes. 

Experimental Treatments ^ 

The following five different sets of experimental conditions 
were represented in the ten classes: 

I an EQUATIONS class in which the IMP Kits were used in 
five class periods during the two-week experiment and 
the regular once-a-week EQUATIONS tournament was continued j 
E an EQUATIONS clasis in which ^he game was played for the 
five class periods without any explicit teaching of the 
21 IMP Kit ideas; 
TE two EQUATIONS classed in which the game wap played for 
five class periods and the teachers expllci^tly taught 
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' the 21 Ideas presented in the IMP Kits; 
0 three non-EQUATIONS clasBes in which the ordinary ctkssroon 
procadure^ was continued without change with no special 
attention given to the 21 fiMP Kit -ideas; and 
T 'fhtee non-EQDATIONS , classes in which the teachers explicitly 
taught the 21 IMP Kit ideas for five class periods. 
The set of conditions pf greatest interest to the researchers was 
that of the I group Iff which' students individually played through 
the IMP Kits, completing as many of the sejt of 105 as they could 
in the five periods*. Before this group started on the IMP Kits, 
one class period was devoted to teaching members of the «clas'8 how 
to use the kits . 

Dependent Variables . 

. \' ^ ■ ' • 

The effects of the various set3 of experimental conditions 
were measured by two different forms of a pair of specially- 

'Constructed tests targeted at the 21 mathematical ideas presented j 
in the IMP Kits. The^ first of the pair of* tests is called a ' 
C test; it contains only G-type items. The second test J.s c|t$led 
an R test; it contains only R-type items. Two different forms of 
the C test were used (Form C and Form D) , as well as two different; 
forms of the R tes^(Form E and Fbm^^F). |In each of th% ten classes 
in which theqe tests. we're administered, the students wer:e divided 
into eight groups — Gl,^, G2, ... G8. Each studentj^ received a C test 

• and an R test as^retests, and each received alternative forms of 
the two testsi as pbsttests as foLldws: 
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Using Ca (after) to denote the score on the* G posttest and Cb (^fore) 
to denote the C pretest saore (and similarly^ for the R pretests and 
posttests), outiRome measures of three dependent variables can be 
specified as follows: \ ^ 

1. ^ Ca - Cb Increase in> performance on C test 

2. Ra - Rb Increase in performance on R test 

3. (Ca-Ra) - (Cb-^Rb) Decrease in difference in performance 

on C te^and R test 

RESULTS^ 
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The scores for each of .the five experimental groups, summarized 
in Figure 1, were significantly higher (at the .0001 level) on the 
C test than on the R test both on pretests and on^osttests. The 
mean pretest score for all students on the C test was 5.10, while 
for the R test it was l.AO (maximum score « 21) a ratio of 'about , 
3.6 to 1. On the postests the ratio ^decreased fo 2.6 to l,with ^ 
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mean scores of 6.32 and 2.A3, respectively. / 

Three of the experimental groups had significant differences" 
bel:ween pretest and Ippstt^st scores oh the C test ' (measured by 
Ca^^Cb) . The IMP Kit gtoup (I) had a mean pretest of 7.70 and 9;26 
on the posttest, significantly higher at the ,001 level, and the 
groups that were explicitly^ taught the 21 ideas (TE and T) went 
from 5.33 to 7»4'8 and from 4.d2v^d 5.32, respectively, sigrtlficantly 
higher dn the posttest at the .OOO!^ level. 

The same three experimental groupSv had significant differences 
between pretest and posttest scores on the R^est (measured by 
Ra-Rb) . The IMP Kit group arid the T group were significantly higher 
on the posttest (.0001) with mean pretest-to pos;tteet scores of 
3 #39 to 6.78 and 0.59 to 1,41, respectively, yJiereas the TE group 
was significaptly higher at the .0005 level with scores of 1.65 
to 2.84. ^ 

Only two of the experimental groups showed significant changes • 
in the gap between C-sense understanding and R-sense unde]a3tanding 
from the pretests to the" posttests (as measured by (Ca-Ra) - (Cb-Rb)) . 
The IMP Kit group achieved a 1.83 reduction in its CR Gap, significant 
at the .005. level, in mo^ing'from a pretest gap of 4.30 to a posttest 
gap of 2.48. On^the other tiand, the T group Increased its CR Gap 
by 0.95 i significant:, at the .05 level, with a pretest gap of 3.67 
and a posttest gap of 4.64. 

In compearing the test scores^ of thg experimental groups with 
each other, onAy those pairs in the total collefitipn that qualify 
by the highly conservative Scheffe procedure (see Winer, 1971) at 
the .05 level of significance ar^e reported as tieing significantly 
different. The results of the between-group comparisons are summarised 
in Figure ,2. . ^ 

On the pretest scores seven of the pairs of groups were 
sigrtlficant;J.y different on the C tests, six pairs were different on 
the R tests, but none weregigni^ise&itly different on the amount of 
the CR Gap. The significance, >i^nes written over the^ names of each 
of the experimental groups l|f^Figtire 2 should be interpreted as 
foliows:. ,.;# . . ^ 

I j "1, Groups whose cnanijls- do not appear under a common line do 

I V differ slgnifl<^te^ from each other; 

j 2. those whose n^ifes do appear under a common line do 'not s 

^ ^ differ sigri^^^filcantl^ from each other. • ' 

fil^^riq^i on C pretest r^^^ the^IMP Kit group was significantly higher 
'^3^1^?^ TE, 0, BXi0t groups (at the ,0001 level); the E grl^p was 
hip^^^^^^ thap the TE group and higher (.0001) than the 0 and 

and thet^fe group was higher (.0005> than the T group. On 
pi|Eli^^test sc<k%s the IMP Kit group was again s$ipiificantly higher ' 
0, and T groups (.0001); the E group, higher thaii the 
f^^^ and the TE grot^p, higher- than the T group 

(^dpOS^^^^^^^ 3.ack of any significant difference on the pretest 
sc9i^e8?^bl^fe^^^ of the pairs of experimental groups with respect 

to thej^^lmdtot CR G^g^is indicated by the appearance of all of 
the nai%s-^of the groups liaiaer a common ^line . • ' 

V Oniithe J^^^ score^ one less pair of groups waa different 

onj i|lj|B (|vtie;^t; one more-p^r, different on, the R test; aSid there 
emeie|;^4 time a pair different wfth respect to the CR Gap. 
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Mathematics Pi^etest and Posttest Scores on C Tests and R Tests 
of Five Experimental Groups of Eighth-Grade Classes 
Pelhkm Middle School, Detroit, 1974 



B 



On the C posttest scores both, the IMP Kit group and the k group 
were significantly higher (.0001) thin the T and 0 groups, and the 
TE group was higher (.0005) than the T group as well as higher 
(.0001) th^ the 0 group. On the R posttest scores the IMP Kit 
group was significantly higher (.0001) than every one of the 
other groups; the E group, higher (.0001) than the T and 0 groups; 
and the TE group, higher (.001) t^ian the T group and higher (.0001) 
than the 0 group. With respect to the amount of CR Gap dn posttest 
scores, the one pair significantly diffjerent resfilted primarily 
from the large pretest-posttest imptoyement in the R test score 
of the IMP Kit group; thus the qR Gap of the 5MP Kit group turned 
out to be iBigniflcantly smaller (.0001) than that of the TE group. 

On the improvement Indicated by the dif fexeuce between ' 
posttest and pretest score?, the- iftp Kit group clearly emetg^es as ^ 
the group that achieved the greatest improvement;* The only other • 
gjoup that was significantly higher than any of the other groups 
on any of the three improvement measures was the TE group. In 
improvement on C test scores only one pair of groups was signifi- 
cantly different:, the improvement of the TE grou^ was greater 
(..005) than that of the 0 group. In improvement on R test scores 
the IMP Kit group w^s significantly greater (.0001) tha» every one. 
of the other^roups , and the TE group was greater (.001) than the / X 
0 group. WitH respect to*whBt is probably the most important 
measure of all — therextent of the improvement in reducing the 
CR Gap the IMP Kit group is the only group significantly better 
thaia any of the other groups. It deserves emphasis that the IMP 
KiMlgroup turned out significantly better on this measure than 
^veiy other group except the E group — and better than the E group, 
al^^Ugh not significantly so. The improvement of the IMP Kit group 
in ;^<lucing the OR Gap was greater (.0005) than thai of the 0 and 
T g^hixpk and greater (.0001) than that of the TE grdup. 

The significant differences on the C pretest arid R pretest 
scores amoiig the experimental groups deserve close scrutiny. Most 
(10 of the 13) of the differences are differences between EQUATIONS , 
groups and nonEQUATIONS groups, and nearly half (6 of 13) are 
differences between the IMP Kit group and other groups. This 
raises the question as to whether the EQUATIONS groups generally 
and the IMP Kit group .in particular were not simply more capable 
students at the beginning of the experiment. If so, perhaps it 
^is not surprising that they improved more in learning to apply / ^ 
''matliematical ideas during the two'-veek experiment. The next 
question is: Given that the students in the EQUATIONS groups were 
more capable at the end of their eighth-grade year when the 
expef^ment was conducted, were they also more capable two years 
earjtielr when they entered seventh grade? y 

; School records indicate that the Stanford Arithmetic Test - 
Advap^ed (computation) was administered to all entering seventh- 
gradei classes two years earlier and that 112 of the students in 
this study participated. Frdjn the scores rejcorded for this sample 
of 237 students in the study, for whom there is this indication 
of m^athematical capability* at the time of entry to tihe sevenths grade , 
It appears that there was no significant difference between any 
of the pairs of the experimental groups at that time. In particular^ 
there was no significant difference between, the IMP Kit group and any 
Of the of her groups. Also, when data for the three EQUATIONS groafm 
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'Significance Levels- of Pairs of Sets o£ Conditions : ^ 
' that Significantly Differ from Each Other on Te^t Scores* 

* J' 

* The only pairs of sets of conditions shown in 'this table as 
significantly different from each other are those ini the 
collection &f pairs that satisfy the higljly conservative 
Scheffe procedure at the .05 level* 



are combined and those for the two nonEqUATIONS groups are also 
combined, there is no significant difference in mean scores between^ 
the JEQUATIONS* gtoupfi and the notiEQUATIONS groups . The data are 
summarized in Table 1. ^ 

; Table 1 ' * 
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Mathematical Capabilities Two Years Earlier: 
Scdres on Stanford Arithmetic Test - Advanced (computation) 
Administefed in September 197?, for 112 of the 237 Students in 
This Study Enrolled in Pelham Middle School Eighth Grade Classes 

May 1974 

The EQUATIONS and nonEQUATIONS groups were quite different 
tvJo school -years later when this experiment was undertaken, as was 
•.the IMP Kit group compared to all other groups" except the E group. 
On the C pretest the EQUATIONS groups had a me«m score of 6.38, 
significantly higher at the .0001 level than the 4.16 mean Score of 
the nonEQUATlQNS groups. On the R pretfest the EQUATIONS groups 
were also significantly higher (.0001); the mean Scores -were 2.36 to 
0.71'. With fespect to the CR Gap the 0.50 difference between the 
me^ans of the two groups was not s jLgnif leant . > The data for the , 
EQUATIONS groups compared to the nonEQUATIONS grempp.^re summarieed 
in Table 2. JThe data for . the other- comparisons are in Figure 2 . 



Table 2 



Test 



Group 



C 

N 
X 

, S:t 
Signli . 

R 

N 

X 
Sx 

Signi^ 
C-R 

N 

• Sx 
Signif . 



Pretest 
■ a 



EQ nonEQ 



87 119 
6.38' 4.16 
2.62 1.94 
.0001 



86 121 
2.36 0.71 
1.94 , 1.08 
.0001 , 



86 116 
4.02 3.52 
2.06 1.72 
ns 
(.0589) 



Posttest 
b 



EQ nonEQ 



84 ■ 118 
8;14 3.02 
3.35 2.41 
.0001 



84 118 
4.18 1.18 
3.17 . 1.55 
.0001 



84 114 
3.96 3.91 
2.44 2:06 
ns 



Posttest - Pretest 
b - a 



EQ ♦ 



nonEQ 



84 • 
1.68, 
2.44 



83 
1.72 
2.20' 



83 

-0 .04 . 
2.99 



.05 



104 
0.91 
2.24 



107 
0.50 
1.19 

.OOO'l 



98 

0 .'41 
2.|2 

ns i* 



Comparison of EQUATIONS and nonEQUATIONS Groups^. 
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DISCUSSION 



This study provides strong support for the proposition that 
skills in applying mathematical ideas can be improved by learning 
procedure^ that are rich in opportunities ' for such application at 
appropriate levels of cpmplexlty for each student. Interpreted 
vio^t favorably, the results show thkt the combination of playing 
EQUATIO^IS over a two-year period and then working intensively with 
the IMP Kits for two weeks enables -students to apply mathematical 
ide^as (in the sense ^tudied in this experiment) better than any 
of the dther four sets of conditions do: better than Just playing 
EQUATIONS alone; better than playing EQUATIJWS and being taught 
explicitly by the teacher the 21 'ideas presented in the IMP Kits, 
better than being taught the 21 ideas in/^ ordinary traditional 
^ jnathematics class, and better thap being in ah Ordinary traditional 
I class without any special teaching ox the ideas — and, furthermore, 
better in each case by a'highly Qop(servative te^t at an extreme 
level of significance (.0001). It should be acknowledged immediately *^ 
✓ that there are some questions with respect to this most favorable ^ 
interpretation that require further investigation. At the start 
of ''the -experiment the IMP Kit group was cleerly performing at a 
higher level of achievement than were the^ other groups. This 
superior performance seems linked-.to theif two-year experience in 
* playing EQUATIONS. The available evidence indicates that the RQUATIdNS, 

and nonEQUATIONS groups were-tiot^tfferent upon their entry to 
the seventh grade, but that after two years' of different experience 
with respec^ to whether or not they played EQUATIONS, the EQUATIONS 
.group was significantly better in^ both compyting with and applying^ 
.these 21 liaathematical ideas. The tophasis of the IMP Kit expedience 
is clearly in the direction of impi^oving skills in applying ! , ' ^ 
mathematical ideas, although it doe^ improve both computing ^and ; ^ 
applying. In jterms of reducing the ^ difference between understanding, 
a mathematical idea ii\ the sense of computing correctly with it 
and understanding it in the sense of being able to apply it in a 
' context whqre i^ must be recognized to be relevant (the CR Gap), ^ 
playing through the IMP Kits clearly is more effective than any " , 
of the °other methods tried in thiQ experiment (except possibly • 
' playing EQUATIONS alone, where ^the effect is in the right direction 
but is not significant). 

For classrobbs in which learning to apply mathematical ideas 
is still a problem, the Implications of the findings -of this study 
are obvious: learning environments so structured are- effective — 
and should be u^ed. L 

-In an earlier study (Allen & Main, 1973), the objectives of 
designing strqctured learning environment's of the type studied here 
were- described as the enhancement^^f 1>oth the affective and cognitive 
dlmentions. It wjis shown there that attitude as measured by 
absenteeism was profoundly affected throiS^h the use 5f such ' 
structured learning environments, and the prediction wad ventured 
~ that in intelligent hands the achievement of knowledge shQuld be, 
too. In this inner-city school in Detroit the learning structure 
we axfe designing has been in such hands.. We stilL need to learn 
more, but the efforts there have advanced our understanding, one 
solid step along the way. 
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